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Goals 


Understand OpenStack purpose and use cases 
Understand OpenStack ecosystem 
O history 
O projects 
e Understand OpenStack architecture 
O logical architecture 
O components 
O request flow 
e Get enough theory for hands-on lab 
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What is OpenStack? 


"Open source software for building 
private and public clouds” 
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OpenStack capabilities 


e VMs on demand 
O provisioning 
o snapshotting 

e Volumes 


e Multi-tenancy 
o quotas for different users 
o user can be associated with multiple tenants 


e Object storage for VM images and arbitrary 
files 


MIRANTIS © Mirantis, Inc, 2012. All rights reserved. 


OpenStack History 


July 2010 - Initial announcement 
October 2010 - Austin Release 
February 2011 - Bexar Release 
April 2011 - Cactus Release 
October 2011 - Diablo Release 
April 2012 - Essex Release 
October 2012 - Folsom Release 
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OpenStack Projects 


Nova (Compute) 
Glance (Image Service) 
Swift (Object Store) 
Keystone (auth) 
Horizon (Dashboard) 
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OpenStack Projects: Relationship 
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OpenStack: Deployment Topology 
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OpenStack Projects: Detailed View 
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OpenStack Projects: Communication Types 
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Use Case: provision a VM 
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Initial State 
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Step 1: Request VM Provisioning via UI/CLI 


User specifies VM params: 
name, flavor, keys, etc. and hits 


"Create" button 
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Step 1: Request Provisioning via UI/CLI 


e From Ul 
a. Login to Horizon 
b. Specify params of VM in "Create VM" form 


C. Hit "Create" button 
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Step 1: Request Provisioning via UI/CLI 


e From Ul 
a. Login to Horizon 
b. Specify params of VM in “Create VM" form 


C. Hit "Create" button 


e Under the hood 
a. Form params are converted to POST data 
b. "Create" request initiate HTTP POST request to back- 


end 
m To Keystone if auth token is not cached 
m [onova-api if auth token hasn't expired yet 
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Horizon 


“The OpenStack Dashboard (Horizon) 
provides a baseline user interface for 
managing OpenStack services.” 
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Horizon notes 


e ‘Stateless’ 
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Horizon notes 


o 
e Error handling is delegated to back-end 
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Horizon notes 


e "Stateless" 
e Error handling is delegated to back-end 
e Doesnt support all API functions 
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Horizon notes 


"Stateless" 

Error handling is delegated to back-end 
Doesn't support all API functions 

Can use memcached or database to store 
sessions 
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Horizon notes 


"Stateless" 

Error handling is delegated to back-end 
Doesn't support all API functions 

Can use memcached or database to store 
sessions 

e Gets updated via nova-api polling 
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Horizon internals 


e 2 subprojects 
O horizon - generic Django libraries and components to 
work with REST-based back-end 
O openstack-dashboard - web app itself, with styles, 
locale, etc. 
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Horizon internals 


e 2 subprojects 
O horizon - generic Django libraries and components to 
work with REST-based back-end 
O openstack-dashboard - web app itself, with styles, 
Locale, etc. 


e Dashboard for each entity (like instances or 
images) - nested Django app 
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Step 2: Validate Auth Data 


Horizon sends HTTP request to 
Keystone. Auth info is specified 


in HTTP headers. 
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Step 2: Validate Auth Data 


e From UI 


a. Horizon sends HTTP request to Keystone 


b. Keystone parses HTTP header info and verifies that 
m [hecredentials are valid 


m User-Tenant-Role mapping is valid 
m The requested action is available for this user 
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Keystone 


"Keystone is an OpenStack project that 
provides Identity, Token, Catalog and 
Policy services for use specifically by 

projects in the OpenStack family." 
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Keystone Architecture 


MIRANTIS 
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token identity 
back-end back-end 
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Keystone data model 


e User: has account credentials, is associated with one or more 
tenants 


e Tenant: unit of ownership in OpenStack, contains one or more 
users 


e Role: a first-class piece of metadata associated with many user- 
tenant pairs. 


e Token: identifying credential associated with a user or user and 
tenant 

e Extras: bucket of key-value metadata associated with a user-tenant 
pair. 


e Rule: describes a set of requirements for performing an action. 
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Keystone: auth flow 


Clear existing 
auth headers 


Collect token 
from HTTP 
headers 


Populate 
additional 


Reject the 
valid validate token invalid? request 


(401) 


keystone service unavailable 


Reject the 
request 


(503) 
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Step 2: Validate Auth Data 


e From UI 


a. Horizon sends HTTP request to Keystone 


b. Keystone parses HTTP header info and verifies that 
m The credentials are valid 


m User-Tenant-Role mapping is valid 
m The requested action is available for this user 
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Keystone key concept 


e What service exposes 


o http://myservice/instances/* - GET/POST/PUT 
o http://myservice/images/* - GET 


e How mapping works 
o User X in Tenant Y can do actions A,B,C 
m User: GET/POST/PUT on instances, images 
m Admin: GET/POST/PUT on tenants, users, quotas 
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Keystone: populating auth data 


. Add tenants 

. Add users 

Add roles 

. Grant roles to users 

. Add endpoint templates 

. Map endpoint templates to zones 


O Ui E ui ND EB 
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Step 2: Validate Auth Data 


e From UI 
a. Horizon sends HTTP request to Keystone 
b. Keystone parses HTTP header info and verifies that 
m The credentials are valid 
m User-Tenant-Role mapping is valid 
m The requested action is available for this user 


e |f operation is allowed - send token back 
e |f operation is not allowed - send error code 
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Step 2: Validate Auth Data 


Keystone sends temporary 
token back to Horizon via HTTP. 
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Step 3: Send API request to nova-api 


Horizon sends POST request to 
nova-api (signed with given 


token). 
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Step 5: Send API request to nova-api 
e Horizon sends HTTP POST request to nova-api 


e nova-api doesn't know about Horizon, so it 
o Makes HTTP request to Keystone to validate token 
o |f valid - processes the request 
o If invalid or expired - send back HTTP error 
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Step 4: Validate API Token 


nova-api sends HTTP request to 
validate API token to Keystone. 
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Step 4: Validate API Token 


Keystone validates API token 
and sends HTTP response with 


token acceptance/rejection info. 
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Step 5: Process API request 


nova-api parses request and 
validates it by fetching data from 
nova-db. If request is valid, it 


saves initia db entry about VM 
to the database. 
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Step 5: Process API request 


e Validate request params 
o Type errors are verified on code level 
o Cloud-related params are validated via DB requests 


e |f request cannot be processed then throw an 
exception 


e |f request can be processed 
o Save initial state to the database 
o Send message with next actions to MQ 
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nova-api 


-hova-api is a RESTful API web service 
which Is used to interact with nova" 
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nova-api characteristics 


Exposes REST API 

Provides system for managing multiple APIs 
on different sub-domains 

O EC2-compatible - will be deprecated 

O OpenStack Compute API - all innovation happens here 
The only "allowed" way to interact with nova 
Stateless - HA-ready 
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nova-api characteristics 


e Exposes REST API 
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nova-api characteristics 


e Provides system for managing multiple APIs 


on different sub-domains 
O EC2-compatible - will be deprecated 
O OpenStack Compute API - all innovation happens here 
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nova-api characteristics 


O 
O 


e The only "allowed" way to interact with nova 
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nova-api characteristics 


Exposes REST API 

Provides system for managing multiple APIs 
on different sub-domains 

O EC2-compatible - will be deprecated 

O OpenStack Compute API - all innovation happens here 
The only ‘allowed’ way to interact with nova 
Stateless - HA-ready 


MIRANTIS © Mirantis, Inc, 2012. All rights reserved. 


nova-api clients 


Compute API 
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nova database 


"nova database stores current state of 
all objects in compute cluster.” 
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nova database 


e Can be any relational database 
e nova-api talks to DB via SOLAlchemy (python 


MIRANTIS 


ORM) 

Most of the deployments are done with 
MySQL or PostgreSQL 

DB HA should be done via external tools (like 
MMM for MySQL) 
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Step 6: Publish provisioning request to queue 


nova-api makes rpc.call to 
scheduler. It publishes a short 
message to scheduler queue 


with VM info. 
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Step 6: Pick up provisioning request 


scheduler picks up the message 
from MQ. 
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Message queue 


"Message queue is a unified way for 
collaboration between nova 
components. 
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OpenStack messaging 
2. Send JSON message . 3. Selected receiver fetches 
Component A with receiver ID and entry ID PabbitMQ message, unpacks entry ID Component B 
next expected step and processes itin DB 


nova 4. Gets entry by entry ID 
1. Creates/updates entry for further processing 


2 modes: 
e rpc.cast - dont wait for result 
e rpc.call - wait for result (when there is something to 
return) 
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Messagings notes 


e OpenStack uses multiple queues within single 
RabbitMO instance 
o 1 Queue per service 
o 1 fanout queue 
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Messagings notes 


O 
O 


e OpenStack messages traffic is not intensive 
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Messagings notes 


e OpenStack doesnt send broadcast messages 
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Messagings notes 


e HA for MQ should be configured separately 
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Step 7: Schedule provisioning 


Scheduler fetches information 
about the whole cluster from 
database and based on this info 
selects the most applicable 
compute host. 
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nova-scheduler 


“nova-scheduler is a daemon, which 
determines, on which compute host 
the request should run." 
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nova-scheduler: users demand 


e provision VM to particular host 
e provision VMs of the particular tenant to 


isolated hosts 


e provision all VMs on different hosts 
e provision VMs to ‘higher density" hosts 


MIRANTIS 
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nova-scheduler: available schedulers 


Scheduler [Description O 
Picks a host that is up at random 


Simple Picks a host that is up and has the 
fewest running instances 


Filter Picks the best-suited host which 
satisfies selected filter 

Multi A scheduler that holds multiple sub- 
schedulers 
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nova-scheduler: filtering 


em 
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nova-scheduler: filters 


Fier peu 
Same host or different host 


availability | Least cost inside selected availability 
zone zone 


Least CPU core utilization 
Only return hosts with sufficient RAM 


Allows simple JSON based grammar. 
Can be used to build custom 
schedulers. 
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nova-scheduler: filters 


Filters are statically configured in nova.conf 
Multiple filters can be specified 


It is possible to create custom filter 
o Inherit from BaseHostFilter class 
o override host passes(self, host state, filter properties) 
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nova-scheduler: weights and costs 


colt? St Cost 
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nova-scheduler: weights and costs 


e Cost - integer value 

e Every compute host can have several cost 
functions associated with it 

e |f no cost functions associated - use default 
from nova.conf 

e weight = sum(cost, + weigth fn) 
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nova-scheduler: summary 


e Allow to tweak provisioning by adjusting 
filters, cost and weights 


e Still doesn't cover all customer demands - 
exposes framework for building custom 
schedulers instead 
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Step 8: Start VM provisioning on compute node 


Scheduler publishes message 
to the compute queue (based on 
host ID) and triggers VM 


provisioning 
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VM Provisioning Algorithm 


1. nova-compute gets message from MQ 

2. nova-compute fetches required instance 
information from DB 

5. nova-compute starts instance via hypervisor 


driver 
a. driver requests image from glance by image id 
b. if image is available, driver downloads image (depends 
on implementation) 
c. when download is complete, driver renders image 
4. nova-compute calls nova-network to configure 


networking for created instance 
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Step 9: Start VM rendering via hypervisor 


nova-compute fetches 
information about VM from DB, 
creates a command to 
hypervisor and delegates VM 
rendering to hypervisor. 
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nova-compute 


"nova-compute Is a worker daemon, 
which primarily creates and terminates 
VMs via hypervisor API.” 


MIRANTIS © Mirantis, Inc, 2012. All rights reserved. 


nova-compute 


nova-compute 


e R : 
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nova-compute: drivers 


e Functionality is not 100% similar 

e Exact run instance” flow depends on driver 
implementation 

e Most of the features are tested on KVM 
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nova-compute config 


e --libvirt type 
Hypervisor being used. In this deployment ‘kvm’ is specified. 
e --libvirt uri 
URI to use for connection to hypervisor. In this deployment ‘qemu+tcp:///system’ is 
specified. 
e --sql connection 
Database connection string in SOLAlchemy format. This is used for connecting to state 


database. 
e --rabbit host 

IP address for RabbitMO host. Non-standard port also can be specified 
e --glance host 


IP address and port of Glance Image Service host. This is needed for streaming virtual boot 
images. 

e --glance api server 
IP address and port of Glance API server. This is needed for getting virtual boot images 
meta-data. 
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Step 10: Request VM Image from Glance 


hypervisor request VM image 
from Glance via Image ID 
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Glance 


"The Glance project provides services for 
discovering, registering, and retrieving virtual 
machine images. 
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Glance summary 


e Images-as-a-Service 
e Can use multiple back-ends for image storage 
e Supports multiple image formats 
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Glance architecture 
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Glance capabilities 


MIRANTIS 


CRUD images 

Search images via filters 

o name 

o container format 

o disk format 

o size min, size max 

O Status 

Caches images 

uses SQLite or FS that supports xattrs for caching 
queues images for prefetching 
prefetches images 

prunes images 

cleans invalid cache entries 


O O © © 
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Glance image formats 


Disk Format [Description O 
This is an unstructured disk image format 


This is the VHD disk format, a common disk format used by virtual machine monitors from 
VMWare, Xen, Microsoft, VirtualBox, and others 


Another common disk format supported by many common virtual machine monitors 
A disk format supported by VirtualBox virtual machine monitor and the OEMU emulator 
An archive format for the data contents of an optical disc (e.g. CDROM). 


A disk format supported by the OEMU emulator that can expand dynamically and supports Copy 
on Write 


This indicates what is stored in Glance is an Amazon kernel image 
This indicates what is stored in Glance is an Amazon ramdisk image 
This indicates what is stored in Glance is an Amazon machine image 
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Custom image creation 


1. Get installation ISO 
2. Create VM (qemu-img create) 


5. Start VM and connect to it via VNC console 
a. Install image without LVM 
b. Create default iptables rules 
c. Install and configure cloud-init 
d. With cloud-init configure image 
4. Prepare image for OpenStack 
a. Extract root partition, kernel and ramdisk 
b. cleanup 
C. package 
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Fetching image from glance 


— 


MIRANTIS 


. GET http://<glance-url>/images/<|D> 


If image can be found, API returns image-uri 
nova-compute passes image-uri to hypervisor 
driver 


. hypervisor driver fetches image directly from 


glance back-end store using image-uri 
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Step 11: Get Image URI from Glance 


If image with given image ID 
can be found - return 
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Step 12: Download image from Swift 


hypervisor downloads image 
using URI, given by Glance, 
from Glance's back-end. After 


downloading - it renders it. 
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Step 13: Configure network 


nova-compute makes rpc.call to 
nova-network requesting 


networking info. 
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Network configuration flow 


Allocate MAC addresses 
Allocate IPs (for each network) 
Associate IPs with VMs (DB) 


Setup network on host 

a. Update DHCP config 

b. Initialize gateway 

c. VPN configuration (optional) 


5. Update networking info in DB 


deo d 
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Step 14: allocate and associate network 


nova-network updates tables 
with networking info and VM 


entry in database 
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nova-network 


"nova-network Is a worker daemon 
which performs tasks to manipulate 
network via external commands." 
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nova-network responsibilities 


e Allocate and configure network via network 
manager 
o FlatManager 
o FlatDHCPManager 
o VlanManager 
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nova-network responsibilities 


O 
O 
O 


e Manage Floating IPs 
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nova-network responsibilities 


o 
e Manage Security groups 
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Network manager 


e Determines network layout of the cloud 
infrastructure 
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Network manager 


e Determines network layout of the cloud 
infrastructure 


e Capabilities of network managers 


MIRANTIS 
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Plugging instances into linux bridges 
Creating linux bridges 

IP allocation to instances 

Injecting network configuration into instances 
Providing DHCP services for instances 
Configuring VLANs 

Traffic filtering 

Providing external connectivity to instances 
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Floating & fixed IPs 


e Fixed IPs: 
o given to each instance on boot 
o private IP ranges (10.0.0.0, 192.168.0.0, etc.) 
o onlyfor communication between instances and to 
external networks 
o inaccessible from external networks 
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Floating & fixed IPs 


O 


e Floating IPs: 


O 


O 


allocated & associated to instances by cloud users 
bunches of publicly routable IPs registered in 
Openstack by cloud dmin 


o accessible from external networks 


MIRANTIS 


multiple floating IP pools, leading to different ISP-s 


© Mirantis, Inc, 2012. All rights reserved. 


CloudPipe 


Compute Node Network Node 


` Project VLAN 


: CI oudpipe i Dnsmasq 
Mava Laara VP DHCP Server 


IPTables 
Source Natting 


IPTables 


Public Virtual Public IP 


Machine 


Compute Node 
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Public Internet 


Security Groups 


All traffic All traffic 
HTTP, FTP, SSH 
HTTP, FTP, SSH Ports 1-1024 


ZER 
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Security Groups 


MIRANTIS 


Security group is a named collection of 
network access rules 

User can select multiple security groups 
during VM creation 

If no security groups specified - default is 
selected 

Security groups are applied on the host node 
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Step 15: Request volume attachment 


Tenant is created, provisioning 
quota is available, user has an 


access to Horizon/CLI 
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nova-volume 


"nova-volume manages the creation, 
attaching and deattaching of 
persistent volumes to compute 
instances” 
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nova-volume summary 


e Optional 
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nova-volume summary 


o 
e iSCSI solution which uses LVM 
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nova-volume summary 


o 

o 

e Volume can be attached only to 1 instance at 
a time 
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nova-volume summary 


e Persistent volumes keep their state 
independent of instances 
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nova-volume summary 


e Within single OpenStack deployment different 
Storage providers cannot be used 
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nova-volume drivers 


iSCSI 

Xen Storage Manager 
Nexenta 

NetApp 

SAN 


MIRANTIS © Mirantis, Inc, 2012. All rights reserved. 


Initial State 


Tenant is created, provisioning 
quota is available, user has an 


access to Horizon/CLI 
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Request Flow Retrospective 
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